Blurring or degradation of an image can be caused by various factors, e.g. out-of-focus optics, or any other aberrations that result from the use of a wide-angle lens, or the combination of inadequate aperture value, focal length and lens positioning. During the image capture process, when long exposure times are used, the movement of the camera, or the imaged subject, can result in motion blurring of the picture. Also, when short exposure time is used, the number of photons being captured is reduced, this results in high noise levels, as well as poor contrast in the captured image.
Various methods for restoring images that contain defects, e.g. blurring, are known from related art. For example spatial error concealment techniques attempt to hide a defect by forming a good reconstruction of the missing or corrupted pixels. One of the methods is to find a mean of the pixels in an area surrounding the defect and to replace the defect with the mean pixel value. A requirement for the variance of the reconstruction can be added to equal the variance of the area around the defect.
Different interpolation methods can also be used for restoration. For example a bilinear interpolation can be applied to pixels on four corners of the defect rectangle. This makes a linear, smooth transition of pixel values across the defect area. Bilinear interpolation is defined by the pixel value being reconstructed, pixels at corners of the reconstructed pixel and a horizontal and vertical distance from the reconstructed pixel to the corner pixels. Another method is edge-sensitive nonlinear filtering, which interpolates missing samples in an image.
The defect block can be replaced also with the average of some of all of the surrounding blocks. One example is to use three blocks that are situated above the defect. Further there is a method called “best neighbours matching” which restores images by taking a sliding block the same size as the defect region and moves it through the image. At each position, except for ones where the sliding block overlaps the defect, the pixels around the border of the sliding block are placed in a vector. The pixel values around the border of the defect are placed in another vector and the mean squared error between them is computed. The defect region is then replaced by the block that has the lowest border-pixel.
The purpose of image restoration is to remove those degradations so that the restored images look as close as possible to the original scene. In general, if the degradation process is known; the restored image can be obtained as the inverse process of the degradation. Several methods to solve for this inverse mathematical problem are known from the prior art. However, most of these techniques do not consider the image reconstruction process in the modelling of the problem, and assume simplistic linear models. Typically, the solutions in implementations are quite complicated and computationally demanding.
The methods from related art are typically applied in restoration of images in high-end applications such as astronomy and medical imaging. Their use in consumer products is limited, due to the difficulty of quantifying the image gathering process and the typical complexity and computational power needed to implement these algorithms. Some of the approaches have been used in devices that have limited computational and memory resources. The methods from the related art are typically designed as a post-processing operation, which means that the restoration is applied to the image, after it has been acquired and stored. In a post-processing operation each colour component has a different point spread function that is an important criteria that can be used to evaluate the performance of imaging systems. If the restoration is applied as post-processing, the information about the different blurring in each colour component is not relevant anymore. The exact modelling of the image acquisition process is more difficult and (in most cases) is not linear. So the “inverse” solution is less precise. Most often, the output of the digital cameras is compressed to .jpeg-format. If the restoration is applied after the compression (which is typically lossy), the result can amplify unwanted blocking artefacts.